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INTRODUCTION 

In  this  paper  we  present  a  scale-independent  method  of 
optimization  with  a  stochastic  global  optimization  approach  introduced 
by  Kennedy  and  Eberhart  [1],  the  Particle  Swann  Optimizer  (PSO). 
We  apply  this  method  to  the  biomechanical  system  identification 
problem  of  finding  positions  and  orientations  of  joint  axes  in  body 
segments  through  the  processing  of  experimental  movement  data  [2,3]. 
We  compare  its  performance  to  the  BFGS  optimizer  which  falls  under 
a  class  of  optimizers  more  commonly  used  for  this  application. 

Traditionally,  gradient-based  methods  such  as  the  BFGS 
algorithm  have  been  used  to  solve  joint  parameter  identification 
problems,  but  major  drawbacks  to  these  methods  are  their  sensitivity 
to  problem  scaling  and  algorithm  parameter  selection.  These 
drawbacks  require  a  costly  and  time-consuming  parameter  sensitivity 
studies  to  be  carried  out  for  a  problem  before  consistently  acceptable 
results  can  be  obtained.  In  addition,  the  presence  of  noise  in  the  data 
will  often  cause  premature  convergence  to  an  incorrect  solution. 

The  PSO  method  has  some  very  desirable  qualities  that  can  be 
exploited  in  these  types  of  problems.  First,  because  it  requires  no 
gradient  evaluations  and  because  of  the  way  it  is  fonnulated,  the 
algorithm  is  insensitive  to  scaling  of  the  design  variables.  Second, 
because  of  the  algorithm’s  simplicity,  there  are  very  few  parameters  to 
tune,  and  even  these  have  been  shown  to  be  relatively  problem 
independent.  Finally,  the  concurrent  nature  of  the  swarm  algorithm 
lends  it  to  parallelization,  enabling  the  solution  of  problems  that  are 
too  computationally  challenging  for  single-processor  machines.  The 
need  for  greater  computational  power  is  common  in  the  search  for 
more  realistic  and  accurate  engineering  models  [4],  which  currently 
can  only  be  addressed  by  the  use  of  parallel  algorithms. 

METHODOLOGY 

For  our  test  problem,  we  have  elected  to  use  a  2  degree-of- 
freedom  (DOF),  3-dimensional  ankle  kinematic  model  requiring  12 
parameters  [2],  This  model  is  used  to  generate  synthetic  trajectories  of 
markers  fixed  to  the  foot  and  shank  (three  per  segment).  An 
optimization  approach  was  then  followed  to  recover  the  original  joint 
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parameters  from  this  synthetic  marker  trajectory  data.  One  of  the 
primary  reasons  for  this  approach  is  that  any  solution  found  by  either 
algorithm  can  be  quantified  in  terms  of  final  design  variable  errors. 
Therefore,  we  are  able  to  make  an  immediate  evaluation  of  the 
performance  of  the  optimization  algorithm  that  was  used.  In  addition, 
we  have  control  over  the  magnitude  and  other  characteristics  of  any 
numerical  noise  we  introduce  into  the  system  to  emulate  experimental 
measurement  errors,  and  we  are  able  to  observe  the  impact  varying 
any  of  these  has  on  the  optimization  algorithm. 

The  unconstrained  optimization  (or  system  identification) 
problem  can  be  stated  as  follows: 
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where  x  is  a  proposed  joint  position  and  orientation  in  the  body 
segment,  which  will  have  a  corresponding  marker  configuration  c(x). 
The  fitness  (Eq.  1 )  of  this  marker  configuration  is  evaluated  according 
to  how  closely  it  can  be  aligned  (in  three  dimensions  j)  to  all  of  the  m 
markers  in  configuration  m(t)  over  all  n  recorded  time  frames  t.  This 
matching  or  alignment  of  proposed  virtual  to  observed  marker 
locations  is  done  in  a  separate  optimization  step  (Eq.  2)  by  means  of  a 
non-linear  least  squares  fit,  where  p  is  an  alignment  operator  [5], 

In  order  to  demonstrate  the  potential  sensitivity  to  scaling  in  both 
PSO  and  BFGS  methods,  the  ankle  joint  identification  problem  was 
first  defined  using  the  original  units  of  cm  and  radians  for  the  location 
and  orientation  design  variables  respectively.  Bounds  on  these  design 
variables  were  chosen  to  enclose  a  physically  realistic  interval  around 
the  solution  point  in  the  design  space. 


2003  Summer  Bioengineering  Conference,  June  25-29,  Sonesta  Beach  Resort  in  Key  Biscayne,  Florida 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
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The  scaled  version  of  this  problem  was  then  obtained  by  normalizing 
all  12  variables  to  be  bounded  within  [-1,1]. 

Numerical  noise  was  introduced  by  means  of  superimposing  a 
sine  wave  with  a  random  period,  phase,  and  amplitude  (limited  to  a 
maximum  of  1  cm)  onto  the  marker  data.  This  was  done  to  emulate 
artifacts  caused  by  skin  and  soft  tissue  movement  and  by  camera 
resolution  limitations  found  in  real  data. 

In  generating  the  synthetic  marker  trajectory  data,  we  endeavored 
to  simulate  real  life  data  which  are  analyzed  in  exactly  the  same 
manner.  As  such,  the  problem,  while  still  being  analytical,  required  the 
use  of  parallel  processing  due  to  the  sheer  amount  of  data  to  be 
processed.  Both  the  PSO  and  the  unconstrained  BFGS  gradient-based 
method  were  parallelized  and  evaluated  on  a  cluster  of  29  Linux  based 
PCs  in  the  UF  HCS  Research  Laboratory  (1.33  GHz  Athlons  with 
256MB  memory  on  a  100Mbps  switched  Fast  Ethernet  network). 


RESULTS 


Before  using  the  parallelized  VisualDOC  (Vanderplaats  R  &  D, 
Colorado  Spring,  CO)  implementation  of  the  BFGS  algorithm,  a 
parameter  sensitivity  study  was  performed  in  order  to  obtain  the 
optimum  initial  forward  finite  difference  and  termination  parameters 
of  10"”  and  10"5  respectively.  For  the  parallel  PSO,  the  standard  general 
recommended  parameters  were  used  [6], 

A  total  of  10  unsealed  and  10  scaled  (or  normalized)  optimization 
runs  were  performed  for  both  the  PSO  and  BFGS  algorithms  (Table 
1).  For  both  methods,  the  optimizations  were  started  at  randomly 
chosen  points  within  the  bounds.  The  same  starting  point  locations 
were  used  throughout  all  of  the  runs  when  switching  to  the  scaled 
problem  in  order  to  obtain  a  fair  comparison  for  both  algorithms. 

The  BFGS  algorithm  converged  prematurely  in  all  10  unsealed 
runs,  never  obtaining  even  approximately  correct  joint  positions  and 
orientations,  resluting  in  poor  fitness  values  (large  cumulative  marker 
errors).  Restarting  the  algorithm  at  the  termination  points  yielded  no 
improvement.  After  scaling  the  problem,  however,  we  obtained 
convergence  in  7  out  of  the  10  runs  to  an  approximately  correct 
solution.  Again,  all  attempts  to  further  improve  these  by  a  restart  of  the 
algorithm  are  unsuccessful. 

In  contrast,  with  the  PSO  method,  both  unsealed  and  scaled 
optimizations  produced  a  final  fitness  errors  on  the  order  of  the  noise 
level  with  very  little  standard  deviation  as  compared  to  the  BFGS 
results.  However,  this  reliability  and  accuracy  came  at  a  high  cost  in 
terms  of  function  evaluations  (see  Table  1). 

When  comparing  fitness  value  drift  between  the  scaled  and 
unsealed  problems  during  the  optimization  (Figure  1),  we  observed 


only  a  minute  difference  for  the  PSO  algorithm,  on  the  order  of  10‘4, 
due  to  numerical  round-off  and  truncation.  In  contrast,  the  scaled 
BFGS  solution  very  quickly  diverged  from  the  unsealed  solution  and 
in  all  but  two  cases  terminated  at  an  entirely  different  result  than  found 
for  the  unsealed  problem.  In  these  two  cases,  both  the  scaled  and 
unsealed  optimizations  failed  to  converge  to  the  approximate  solution. 


Algorithm 

Unsealed 

Scaled 

Mean  fitness  error 

BFGS 

3806  ±2477 

889 ± 1601 

±  std.  dev. 

PSO 

69.28±  4 

69.31  ±2 

Mean  function  evals 

BFGS 

PSO 

65 

19700 

305 

20040 

Table  1:  Comparison  of  unsealed  and  scaled  optimization 
results  for  BFGS  and  PSO  algorithms 

CONCLUSIONS 

The  BFGS  requires  scaling  of  the  problem  and  an  extensive 
parameter  sensitivity  study  before  it  is  able  to  find  the  general  solution 
region  consistently.  Even  with  both  these  measures,  it  still  tends  to 
become  trapped  in  local  minima  as  can  be  seen  from  the  large  standard 
deviation  values  in  Table  1.  The  main  advantage  of  this  method  is  its 
efficiency  in  terns  of  function  evaluations.  In  contrast,  the  PSO  is  very 
reliable  in  finding  the  correct  solution  region,  and  is  insensitive  to  both 
the  scaling  of  the  problem  and  initial  algorithm  parameter  selection.  Its 
main  drawback  is  the  high  cost  in  terns  of  function  evaluations 
because  of  slow  convergence  in  the  final  stages  of  the  optimization.  A 
hybrid  approach  could  be  advantageous  if  an  efficient  transition 
criterion  could  be  found  and  optimum  parameters  for  the  BFGS 
algorithm  are  known  in  advance. 
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